REMARKS 

Claims 1-25 are pending. Claims 1-25 are rejected under the 
judicially created doctrine of obviousness-type double patenting based 
on co-pending Application No. 1 0/61 7,422. The claims are also rejected 
under Sections 1 01 ,1 02, 1 03, and 1 1 2. 

Obviousness-Type Double Patenting 

The Applicants of the '422 application are currently in the process 

of considering how that application will proceed in the future. It may be 
that the application is abandoned. This would obviate the double- 
patenting rejection. If not, the Applicants of the present case would be 
amenable to the filing of a Terminal Disclaimer if that is the remaining 
issue in the case, upon the entry of the current Response and Amended 
Claims. 

Claims 7, 1 0, and 22-25 are objected to because of noted 
informalities. Applicants note that the claims have been amended to 
address those objections. 

Rejections Under 35 U.S.C. §112 

Claims 9-25 are rejected under Section 1 1 2. The Examiner notes 
that the claimed subject matter is not described in the specification in a 
way to convey that the Inventors had possession of the claimed 
invention. Applicants respectfully disagree. The Examiner seems to be 
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reading into Claims 9-25 a specific interpretation of those claims that is 
not an exclusive interpretation. Nevertheless, Applicants note that 
Claims 9 and 22 have been amended with respect to the cited peripheral 
devices. Applicants submit that independent Claims 9 and 22 are clear 
and precise in their current form. Therefore, the Section 112 rejections 
of Claims 9-25 should be withdrawn. 

Rejections Under 35 U.S.C. §101 

The Examiner has rejected Claims 1-25, and specifically has 
rejected independent Claims 1 , 9, and 22 as being directed to non- 
statutory subject matter. Specifically, the Examiner takes issue with the 
form of those independent claims, and indicates that as filed, they 
recited non-functional descriptive material or were directed to program 
instructions not limited to a tangible computer-readable medium. 
Applicants submit that Claim 22 has been amended to recite a tangible 
computer-readable storage medium, bearing instructions encoded 
thereon. The storage medium's functionality is certainly realized with the 
instructions encoded thereon, as recited in the claim. Accordingly, Claim 
22 is in a suitable format under the Patent Laws. 

With respect to Claims 1 and 9, those claims specifically recite a 
system with peripheral devices for enabling a system for executing a 
multimodal software application recited in Claim 1 , and recite a method 
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for utilizing peripheral devices to receive inputs and to generate outputs 
to a user when executing a multimodal application. 

Applicants submit that Claims 1 and 9 are further definite, and are 
not simply non-functional descriptive material. Accordingly, Applicants 
submit that the Section 101 rejections have been addressed, and the 
objections should be withdrawn. If there are other helpful suggestions 
that the Examiner may have regarding claim wording for this purpose, 
the undersigned would certainly encourage a telephone conversation on 
the issue, at the Examiner's convenience. 

Rejections Under 35 U.S.C. §102 

Claims 1-3, 7-9, 1 1-13, 15, and 19-20 are rejected under Section 
1 02(b) over the patent of Morin. et al .. U.S. Patent Number 5,748,841 . 
The present invention is directed to a system and methodology for 
executing multimodal software application for interfacing with the 
plurality of peripheral devices that can receive inputs from a user and 
can generate outputs to a user. Specifically, the multimodal 
methodology and system provides a platform for a user to do work by 
interfacing with a user through a plurality of various modalities. To 
operate with the plurality of peripheral devices that provide the plurality 
of different modalities, the present invention is operable for monitoring 
the state of an ongoing dialog, such as from the dialog units that form 
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the workflow description, and selectively adapting the system based on 
the state of the dialog. 

Specifically, while speech, or input through text, such as from a 
keyboard, might be implemented in the present invention, the present 
invention also incorporates a large number of other peripheral devices, 
such as radio-frequency ID readers (RFID), bar code scanners, touch 
screens, printers, etc. As may be appreciated, information may be only 
available from one particular peripheral device, or from a set of 
peripheral devices for a particular dialog unit that defines the current 
state of the dialog. To that end, the present invention monitors the state 
of the dialog for the particular workflow description that is proceeding, 
and selectively enables a particular modality of the input, based upon the 
state of the dialog. That is, the system knows what information is being 
sought based upon the state of the dialog and, in certain situations, will 
not simply rely upon the user or a single hardwired modality. 

Claim 1 recites a first set of peripheral devices for receiving inputs 
from a user in a plurality of modalities, and a second set of peripheral 
devices for generating outputs to a user in a plurality of modalities. The 
multimodal software application receives first data, reflective of an input, 
from the first set of peripheral devices, and outputs second data, to form 
an output, to a second set of peripheral devices. A dialog engine is in 
communication with the multimodal software application, and executes a 
workflow description that has a plurality of dialog units that form a dialog. 
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Claim 1 further recites that the dialog engine selectively enables the 
modality of the input from the first set of peripherals based on a state of 
the dialog of the workflow description. 

Referring to the current published application, Paragraph 70 notes 
that the collection of workflow objects is called a workflow description, or 
dialog. That workflow description, or dialog, is a series of messages to 
output to a user, and includes a number of instances where input is 
expected to be received from a user (Paragraph 75). That dialog can be 
utilized to provide input and output in many different modalities, such as 
speech, audio, scanners, keyboards, touch screens, etc. (Paragraph 75). 
However, while some output is not appropriate for some peripheral 
devices and some input is not going to be provided by certain other input 
devices, each dialog unit within the dialog or workflow description 
includes a designation of which peripheral device is to be used, and 
thus, which modalities are to be used with respect to the associated 
ongoing dialog state or dialog unit. As noted in Paragraph 75, based on 
the state of the dialog, such as a specific dialog unit, the workflow 
description thus reflects what modality of the input is acceptable, such as 
input from a screen, a voice response, or via a bar code scanner, for 
example. As such, the dialog engine executing the workflow description 
can thus, selectively enable the modality of the input from the peripheral 
devices based on the state of the dialog of the workflow description, as 
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opposed to enabling all modalities in respective peripheral devices or 
requiring that certain modalities are selected by the user. 

Claim 1 further recites that the dialog engine further controls 
outputting of a prompt based on an input state of the enabled modality of 
a first set of peripheral devices. That is, the dialog engine selectively 
enables the modality, and then controls outputting of a prompt, based on 
the input state of the enabled modality. The present invention does not 
want to interrupt the dialog when input might be received from a 
peripheral device not even involved in that dialog. The present invention 
addresses that issue as well, by controlling outputting of a prompt from 
the dialog, based on the input statue of the enabled modality of the 
peripheral devices. 

The reference of Morin. et al . does not at all teach a dialog engine 
that selectively enables the modality of the input from a set of peripheral 
devices based on a state of a dialog of the workflow description. 
Furthermore, Morin. et al . also does not teach that the dialog engine 
then controls outputting of a prompt from the workflow description based 
on an input state of that enabled modality. 

First, the Morin. et al . reference does not teach a system for doing 
general work, or executing a workflow description, or dialog, for 
accomplishing that work. Primarily, Morin. et al . is directed to a system 
that allows a user unfamiliar with a computer program language or 
commands to progressively build sentences, which will have meaning to 
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the application or computer program. Thus, the dialog of IVIorin. et al . is 
directed to receiving user inputs and building such sentences based 
upon those inputs received from the user. The multimodality of Morin. et 
al. is generally limited. The system is multimodal only in the sense that it 
provides a mechanism for the user to enter either text or speech as 
input. However, with Morin. et al .. that input technique (text or speech) 
is selected by the user. Then the system proceeds with that selected 
modality to get inputs from a user and to act accordingly. 

The Morin. et al . system does not teach monitoring a particular 
dialog, or specifically the state of a dialog of workflow description to 
selectively enable the modality of the input. The input of Morin. et al . is 
selected as being either text or speech, and a user of the Morin. et al . 
system proceeds accordingly. For example, it is clearly noted in 
Columns 16-22 of Morin. et al . that the operations that are performed 
while in text mode are different from the operations performed while in 
speech mode. That is, those two modes function separately. Therefore, 
a user has to select the mode that will be utilized. Even if the text and 
speech modes are utilized at the same time, there is still no selective 
enabling of the particular modality of the input based on a state of a 
dialog of the workflow description. That is, the selected mode, or 
modes, used within Morin. et al . does not, in any way, relate to the 
workflow description, or a state of the dialog of the workflow description. 
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As such, Morin. et al. does not teach the selective enablement of the 
modality of the input, as recited in Claim 1 . 

Furthermore, IVIorin. et al . does not teach controlling the 
outputting of a prompt from the workflow description, based on the input 
state of the enabled modality . The language referred to in Columns 8, 9, 
and 10 by the Examiner does not, in any way, teach that feature wherein 
the input state of the enabled modality is monitored, and the output of a 
prompt is controlled based upon that input state. The language referred 
to in Column 10 by the Examiner is directed to a context handler and 
history handler of the dialog of Morin. et al . There is no discussion 
whatsoever with respect to controlling the outputting of a prompt based 
upon what is being input at an enabled modality. Again, there is no 
selective enablement of the modality, as taught in Morin. et al .. and thus, 
Morin. et al . cannot teach controlling the outputting of a prompt based on 
the input state of the enabled modality. 

With respect to Column 9 referred to by the Examiner, that 
column does not at all teach controlling outputting of a prompt. There is 
a discussion in Column 9 with respect to managing a dialog and 
interpreting user inputs or initiating questions to the user when 
information is needed. However, there is absolutely no teaching with 
respect to selectively controlling the outputting of a specific prompt 
based on an input state of the enabled modality of a set of peripheral 
devices. 
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Finally, with respect to Column 8 of Morin, et al , that column 
merely refers to generating output messages to the user. There is no 
discussion therein in Lines 1 7-37 regarding selectively controlling the 
outputting based upon monitoring and determining an input state of the 
enabled modality of a set of peripheral devices. 

Accordingly, the Morin. et al . reference fails to teach all the 
limitations set forth in Claim 1 , and thus, does not anticipate Claim 1 
under Section 102(b). 

Each of the dependent Claims 2-3 and 7-8 depend from Claim 1 , 
and thus, recite all the limitations therein, and would be allowable for the 
reasons noted above. Furthermore, each of those dependent claims 
recites a unique combination of elements not taught by the cited art. 

Similar to Claim 1 , Claim 9 has been amended to recite a method 
that includes the steps of executing a workflow description, that Includes 
a plurality of dialog units that form a dialog, and selectively enabling a 
modality of the input from a plurality of peripheral devices, based on a 
state of the dialog of the workflow description. Claim 9 further recites the 
step of controlling the outputting of a prompt according to an Input state 
of the enabled modality. As noted above, the Morin. et al . reference 
does not at all teach a method setting forth the steps as recited in Claim 
9. As thus, since Morin . et al . does not teach all of the elements recited 
in Claim 9, that claim is not anticipated under Section 102(b). 
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Claims 12-13, 15, and 19-20 all depend from Claim 9, and thus, 
include the limitations therein. Accordingly, those dependent claims are 
also not anticipated under Section 1 02(b). Furthermore, each of those 
claims recites a unique combination of steps that is not taught by the 
cited art. 

Rejections Under 35 U.S.C. §1 03(a) 

Claims 4-5 and 1 6-1 8 are rejected under Section 1 03(a) over the 
combination of Morin. et al .. as modified by Gerqic. et al. . U.S. 
Publication Number 2002/0198719. 

Claims 4-5 depend from Claim 1 , and Claims 1 6-1 8 depend from 
Claim 9, which are allowable over the Morin. et al . reference for the 
reasons noted above. The Gerqic. et al . reference is relied upon by the 
Examiner for the concept of filling two fields of Information with a single 
utterance. However, Gerqic. et al . does not provide the teaching lacking 
in Morin. et al . such that the combination of the references would render 
obvious Claim 1 or Claims 4 and 5. Accordingly, those claims are in 
allowable form. Similarly, Claims 9 and 16-18 are in an allowable form, 
and are not rendered obvious by the cited combination. 

Claims 6 and 14 are rejected over Morin. et al .. as modified by 
Hanle. et al .. U.S. Patent Number 5,012,51 1 . The Examiner relies on 
Hanle, et al . for the teaching of non-interruptible prompts. However, 
Hanle. et al does not provide the teachings lacking in Morin. et al . noted 
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above, such that the combination of references would render obvious 
Claim 1 or 9. As such, Claims 6 and 14, which respectively depend from 
Claims 1 and 9, would not be rendered obvious by the cited combination. 
Accordingly, those claims are in an allowable form. 

Claims 10, 22-23, and 25 are rejected over the combination of 
Mom, et al . and French-St. George, et al .. U.S. Patent Number 
6,012,030. As noted above. Claim 1 is allowable over Morin et al . 
French-St. George, et al . is recited for its teaching regarding a visual 
control of a GUI screen. However, the reference does not provide the 
teaching missing in Morin. et al . such that the combination of those two 
references would render obvious Claim 1 or Claim 1 0. Accordingly, 
Claim 1 0 is allowable. 

With respect to Claim 22, that independent claim has been 
amended to recite a tangible computer-readable storage medium 
bearing instructions encoded thereon to cause one or more processors 
to perform the steps of executing a workflow description, that includes a 
plurality of dialog units that form a dialog, and selectively enabling a 
modality of an input, based upon a state of the dialog of the workflow 
description. Claim 22 further recites controlling the outputting of a 
prompt according to an input state of the enable modality. For the 
reasons noted above with respect to Claims 1 and 9, the Morin. et al . 
reference does not teach the invention as recited in Claim 22. 
Furthermore, the combination of the French-St. George, et al . reference 
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does not provide the teachings that are lacking in IVIorin. et al . such that 
the two references would teach all the limitations recited in Claim 22. 
Thus, Claim 22 is not rendered obvious by the cited art. Dependent 
Claims 23 and 25 depend from Claim 22, and also recite a unique 
combination of elements not rendered obvious of the combination of 
Mom, et al . and French-St. George, et al . 

Claim 22 is rejected over the combination of Morin. et al . and 
Setlur. et al .. U.S. Patent Number 5,956,675. Setlur. et al . is referred to 
for teaching a barge-in attempt, and does not teach the elements 
missing from Morin. et al . such that a combination of the two references 
would render obvious Claim 21 , which depends from allowable Claim 9. 
Accordingly, Claim 21 is in an allowable form as well. 

Claim 24, which depends from independent Claim 22, is rejected 
over the three-reference combination of Morin, et al./French-St. George, 
et al./Gergic. et al . As noted above, the combination of Morin. et 
al./French-St.George. et al . does not render obvious Claim 22. The 
addition of the Gergic. et al . reference would also not render obvious that 
claim because it does not teach all the limitations that are missing from 
the two-reference combination. Accordingly, Claim 24 is also in an 
allowable form over the cited art. 
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Rejections Under 35 U.S.C. §1 03(b) 

In light of the foregoing, it is respectfully submitted that the 
pending claims are allowable and the present application is in a 
condition for allowance. If the application is in condition for allowance 
otherwise, Applicants will be amenable to a Terminal Disclaimer, if it is 
necessary, as noted. If it is found that the present amendment does not 
place the application in a condition for allowance, Applicants' 
undersigned attorney requests that the examiner initiate a telephone 
interview to expedite prosecution of the application. 

Applicants do not believe that any fees are due in connection with 
this response. However, if any fees are necessary, the Commissioner 
may consider this to be a request for such and charge any necessary 
fees to deposit account 23-3000. 

Respectfully submitted, 

WOOD, HERRON & EVANS, LLP. 

/Kurt A. Summe/ 

Kurt A. Summe 
Reg. No. 36,023 

2700 Carew Tower 

441 Vine Street 

Cincinnati, Ohio 45202 

(513) 241-2324 

(513) 241-6234 facsimile 

ksumme@whepatent.com 
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